The present invention generally relates to improving healthcare application workflow. In particular, the present invention relates to use of gesture recognition to improve healthcare application workflow.
A clinical or healthcare environment is a crowded, demanding environment that would benefit from organization and improved ease of use of imaging systems, data storage systems, and other equipment used in the healthcare environment. A healthcare environment, such as a hospital or clinic, encompasses a large array of professionals, patients, and equipment. Personnel in a healthcare facility must manage a plurality of patients, systems, and tasks to provide quality service to patients. Healthcare personnel may encounter many difficulties or obstacles in their workflow.
In a healthcare or clinical environment, such as a hospital, a large number of employees and patients may result in confusion or delay when trying to reach other medical personnel for examination, treatment, consultation, or referral, for example. A delay in contacting other medical personnel may result in further injury or death to a patient. Additionally, a variety of distraction in a clinical environment may frequently interrupt medical personnel or interfere with their job performance. Furthermore, workspaces, such as a radiology workspace, may become cluttered with a variety of monitors, data input devices, data storage devices, and communication device, for example. Cluttered workspaces may result in efficient workflow and service to clients, which may impact a patient's health and safety or result in liability for a healthcare facility.
Data entry and access is also complicated in a typical healthcare facility. Speech transcription or dictation is typically accomplished by typing on a keyboard, dialing a transcription service, using a microphone, using a Dictaphone, or using digital speech recognition software at a personal computer. Such dictation methods involve a healthcare practitioner sitting in front of a computer or using a telephone, which may be impractical during operational situations. Similarly, for access to electronic mail or voice messages, a practitioner must typically use a computer or telephone in the facility. Access outside of the facility or away from a computer or telephone is limited.
Thus, management of multiple and disparate devices, positioned within an already crowded environment, that are used to perform daily tasks is difficult for medical or healthcare personnel. Additionally, a lack of interoperability between the devices increases delay and inconvenience associated with the use of multiple devices in a healthcare workflow. The use of multiple devices may also involve managing multiple logons within the same environment. A system and method for improving ease of use and interoperability between multiple devices in a healthcare environment would be highly desirable.
In a healthcare environment involving extensive interaction with a plurality of devices, such as keyboards, computer mousing devices, imaging probes, and surgical equipment, repetitive motion disorders often occur. A system and method that eliminates some of the repetitive motion in order to minimize repetitive motion injuries would be highly desirable.
Healthcare environments, such as hospitals or clinics, include clinical information systems, such as hospital information systems (HIS) and radiology information systems (RIS), and storage systems, such as picture archiving and communication systems (PACS). Information stored may include patient medical histories, imaging data, test results, diagnosis information, management information, and/or scheduling information, for example. The information may be centrally stored or divided at a plurality of locations. Healthcare practitioners may desire to access patient information or other information at various points in a healthcare workflow. For example, during surgery, medical personnel may access patient information, such as images of a patient's anatomy, that are stored in a medical information system. Alternatively, medical personnel may enter new information, such as history, diagnostic, or treatment information, into a medical information system during an ongoing medical procedure.
In current information systems, such as PACS, information is entered or retrieved using a local computer terminal with a keyboard and/or mouse. During a medical procedure or at other times in a medical workflow, physical use of a keyboard, mouse or similar device may be impractical (e.g., in a different room) and/or unsanitary (i.e., a violation of the integrity of an individual's sterile field). Re-sterilizing after using a local computer terminal is often impractical for medical personnel in an operating room, for example, and may discourage medical personnel from accessing medical information systems. Thus, a system and method providing access to a medical information system without physical contact would be highly desirable to improve workflow and maintain a sterile field.
Imaging systems are complicated to configure and to operate. Often, healthcare personnel may be trying to obtain an image of a patient, reference or update patient records or diagnosis, and ordering additional tests or consultation. Thus, there is a need for a system and method that facilitate operation and interoperability of an imaging system and related devices by an operator.
In many situations, an operator of an imaging system may experience difficulty when scanning a patient or other object using an imaging system console. For example, using an imaging system, such as an ultrasound imaging system, for upper and lower extremity exams, compression exams, carotid exams, neo-natal head exams, and portable exams may be difficult with a typical system control console. An operator may not be able to physically reach both the console and a location to be scanned. Additionally, an operator may not be able to adjust a patient being scanned and operate the system at the console simultaneously. An operator may be unable to reach a telephone or a computer terminal to access information or order tests or consultation. Providing an additional operator or assistant to assist with examination may increase cost of the examination and may produce errors or unusable data due to miscommunication between the operator and the assistant. Thus, a method and system that facilitates operation of an imaging system and related services by an individual operator would be highly desirable.
Additionally, image volume for acquisition and radiologist review continues to increase. PACS imaging tools have increased in complexity as well. Thus, interactions with standard input devices (e.g., mouse, trackball, etc.) have become increasingly more difficult. Radiologists have complained about a lack of ergonomics with respect to standard input devices, such as a mouse, trackball, etc. Scrolling through large datasets by manually cine-ing or scrolling, repeated mouse movements, and other current techniques have resulted in carpel tunnel syndrome and other repetitive stress syndromes. Radiologists have not been able to leverage other, more ergonomic input devices (e.g., joysticks, video editors, game pads, etc.), because the devices are not custom configurable for PACS and other healthcare application interactions.
Tablets, such as Wacom tablets, have been used in graphic arts but have no current applicability or interactivity with other applications, such as healthcare applications. Handheld devices, such as personal digital assistants or pocket PCs, have been used for general scheduling and note-taking but have not been adapted to healthcare use or interaction with healthcare application workflow.
Devices facilitating gesture-based interaction typically afford motion-based interactions whereby a user writes or motions a character or series of characters that corresponds to a specific software function. Gesture recognition algorithms typically attempt to recognize a pattern or character gestured by the user. Typical gesture recognition systems focus on recognition of the gestured character alone. In the case of an image magnify, a user must gesture, for example, the letter “z.” The gesture-enabled image processing or display system responds by generically zooming the image. Unfortunately, the system is unaware of a specific level of zoom that the user is requesting from this gesture based interaction. If a user would like to further zoom in, he/she must repeatedly gesture the letter “z” to zoom to the appropriate level. Such repetition may not only be time consuming, but may also be a physical drain on the user.
As discussed above, clinicians, especially surgeons, are challenged with maintaining a sterile environment when using conventional computer devices such as a mouse and keyboard. Several approaches have been proposed to address the desire to maintain a sterile clinical environment, such as use of a sterile mouse/keyboard, gesture recognition, gaze detection, a thin-air display, voice command, etc. However, problems remain with these approaches. Voice command and control appears to be a viable solution but, due to proximity issues and presence of multiple people in an operating room providing confusion and interference, use of voice command and control may not be very practical or effective. Use of a thin-air display still suffers from very complex interaction with computer(s) in the clinical environment.
Radiologists traditionally want less and more intuitive interaction with computers for using PACS applications. In most cases, interaction problems are compounded by poor graphical user interfaces for functions such as zooming, cine, window scroll (which may involve a more continuous interaction), etc. In most cases, radiologists use a regular mouse or a scroll mouse and experimentally attempt to vary the speed/velocity of scroll/cine, etc.
A graffiti character set may be used with a user interface to allow a radiologist to directly interact with PACS by drawing/writing graffiti characters/gestures on an image and thereby provide a user interface without a separate graphical user interface. However, for zooming, scrolling or cine, users will have to write the corresponding characters multiple times, adding complexity to the process.
A hanging protocol is a set of display rules for presenting, formatting and otherwise organizing images on a display device of a PACS workstation, for example. A display rule is a convention for presenting one or more images in a particular temporal and/or spatial layout or sequence. For example, a hanging protocol may include a set of computer-readable instructions (or display rules, for example) that direct a computer to display a plurality of images in certain locations on a display device and/or display the plurality of images in a certain sequence or order. In another example, a hanging protocol may include a set of computer-readable instructions that direct a computer to place a plurality of images in multiple screens and/or viewports on a display device. In general, a hanging protocol may be employed to present a plurality of images for a diagnostic examination of a patient anatomy featured in the images.
A hanging protocol may direct, for example, a PACS workstation to display an anterior-posterior (“AP”) image adjacent to a lateral image of the same anatomy. In another example, a hanging protocol may direct PACS workstation to display the AP image before displaying the lateral image. In general, a hanging protocol dictates the spatial and/or temporal presentation of a plurality of images at PACS workstation.
A hanging protocol differs from a default display protocol (“DDP”). In general, a DDP is a default workflow that applies a series of image processing functions to image data. The image processing functions are applied to the image data in order to present an image (based on the image data) to a user. The image processing functions alter the appearance of image data. For example, an image processing function may alter the contrast level of an image.
DDPs typically include processing steps or functions that are applied before any diagnostic examination of the images. For example, processing functions may be applied to image data in order to enhance features within an image (based on the image data). Such processing functions can include any software-based application that may alter a visual appearance or representation of image data. For example, a processing function can include any one or more of flipping an image, zooming in an image, panning across an image, altering a window and/or level setting in a representation of the image data, and altering a contrast and/or brightness setting in a representation of the image data.
DDPs are usually based on a type of imaging modality used to obtain the image data. For example, image data obtained with a C-arm imaging device in general or a particular C-arm imaging device may have a same or similar DDP applied to the image data. In general, a DDP attempts to present image data in a manner most useful to many users. Conversely, applying a hanging protocol to image data does not alter the appearance of an image (based on the image data), but instead dictates how the image(s) is(are) presented, as described above.
Hanging protocols are currently created using one or more of a mouse, keyboard and graphical user interface (“GUI”) components. Such components are used to specify an image box type (for example, stack, sheet, volume, etc.). The components are used to specify parameters for image sets to be loaded in the image box. The components are also used to set image view presets and/or presentation state, for example. Current hanging protocol creation involves multiple input devices, and a user must switch between keyboard and mouse to create a hanging protocol.
Thus, there is a need for systems and methods to improve healthcare workflow using gesture recognition and other interaction. Furthermore, systems and methods for more streamlined gesture-based control would be highly desirable. Systems and methods for improved creation and management of hanging protocols would also be highly desirable.